Tool is working as designed?

Noticed in IBM DOORS notifications this one

http://www-01.ibm.com/support/docview.wss?uid=swg1PK88530&myns=swgrat&mynp=OCSSYQBZ&mync=E

(PK88530: EXCEPTION ERROR MESSAGE AFTER RUNNING A DXL SCRIPT TO PRINT THE DEFAULT
VALUE OF AN ENUM ATTRIBUTE)

Tested on DOORS 9.2.0.3 this code:
  

Module m = current
AttrDef ad = null
 
for ad in m do
   {
     if (ad.defval)
        {
          print ad.name ": " ad.defval "\n"
        }
   }

 


If you open Edit / Attributes dialog after running this DOORS will crash (even this latest version) with EXCEPTION_ACCESS_VIOLATION. OK, that happens and the exception is that
this is a bug which will be corrected. But the response in the above APAR (?) is:

"Tool is working as designed. Explained to the customer. APAR
rejected"

So, they have a requirement that DOORS should crash after this kind of action?

 


SystemAdmin - Wed May 05 04:19:00 EDT 2010

Re: Tool is working as designed?
SystemAdmin - Wed May 05 06:35:55 EDT 2010

I saw that IBM TechNote as well and was equally confused - are there any IBM people out there such as sammyc who could tell us how such an APAR can be rejected? Can we assume that "ooops" is the answer? :-)

Paul Miller

Re: Tool is working as designed?
Mathias Mamsch - Wed May 05 06:56:21 EDT 2010

Yeah one of the little DXL oddities.

You might not spot it right there, but the below script messes up DOORS, so the problem is not DOORS but the script below. There are a lot of ways to crash DOORS by a script, one needs to be careful when coding. In short: The problem is ambiguity!

Lets see why: The perms that are causing this mess are:
  

ADADefault_ defval ()
ADABool_ defval ()
 
real ::. (AttrDef, ADADefault_)
int ::. (AttrDef, ADADefault_)
string ::. (AttrDef, ADADefault_)
Date ::. (AttrDef, ADADefault_)
bool ::. (AttrDef, ADABool_)
 
string ::.. (real, string)
string ::.. (int, string)
string ::.. (string, string)
string ::.. (Date, string)
string ::.. (AttrBaseType, string)
string ::.. (bool, string)

 


The problematic thing about the script is the line:

 

  

print ad.name ": " ad.defval "\n"



This line can be read as various combinations of the above perms. defval can return either ADADefault or ADABool_. If defval returns ADADefault ad.deval can either be real, int, string or Date. If it returns ADABool_ it will return a bool (read that again). Ok this far?

So the string concatenation ad.deval "\n" will either be a real a bool, a date or a string concatenated to a string.

And now the question: How should the parser know which variant the user wants? It can't. At least he could choose a valid combination, that is if for ad.deval he chooses "Date" he should also call ..(Date, string) but that is where it goes wrong and messes up memory. It will let ad.defval return a Date or and then call the ::..(int, string) function to concatenate the Date to the string.

So after all, what is the solution? Code it like this and the crash goes away.



 

 

  

Module m = current
AttrDef ad = null
 
for ad in m do
   {
     if (ad.defval)
        {
        string s = ad.defval
            print ad.name ": " s "\n"
        }
   }


Hope this was understandable, the topic is a little awkward to explain. Regards, Mathias

 

 

 

 

Mathias Mamsch, IT-QBase GmbH, Consultant for Requirement Engineering and D00RS

 

 

Re: Tool is working as designed?
SystemAdmin - Wed May 05 07:21:02 EDT 2010

Mathias Mamsch - Wed May 05 06:56:21 EDT 2010

Yeah one of the little DXL oddities.

You might not spot it right there, but the below script messes up DOORS, so the problem is not DOORS but the script below. There are a lot of ways to crash DOORS by a script, one needs to be careful when coding. In short: The problem is ambiguity!

Lets see why: The perms that are causing this mess are:
  

ADADefault_ defval ()
ADABool_ defval ()
 
real ::. (AttrDef, ADADefault_)
int ::. (AttrDef, ADADefault_)
string ::. (AttrDef, ADADefault_)
Date ::. (AttrDef, ADADefault_)
bool ::. (AttrDef, ADABool_)
 
string ::.. (real, string)
string ::.. (int, string)
string ::.. (string, string)
string ::.. (Date, string)
string ::.. (AttrBaseType, string)
string ::.. (bool, string)

 


The problematic thing about the script is the line:

 

  

print ad.name ": " ad.defval "\n"



This line can be read as various combinations of the above perms. defval can return either ADADefault or ADABool_. If defval returns ADADefault ad.deval can either be real, int, string or Date. If it returns ADABool_ it will return a bool (read that again). Ok this far?

So the string concatenation ad.deval "\n" will either be a real a bool, a date or a string concatenated to a string.

And now the question: How should the parser know which variant the user wants? It can't. At least he could choose a valid combination, that is if for ad.deval he chooses "Date" he should also call ..(Date, string) but that is where it goes wrong and messes up memory. It will let ad.defval return a Date or and then call the ::..(int, string) function to concatenate the Date to the string.

So after all, what is the solution? Code it like this and the crash goes away.



 

 

  

Module m = current
AttrDef ad = null
 
for ad in m do
   {
     if (ad.defval)
        {
        string s = ad.defval
            print ad.name ": " s "\n"
        }
   }


Hope this was understandable, the topic is a little awkward to explain. Regards, Mathias

 

 

 

 

Mathias Mamsch, IT-QBase GmbH, Consultant for Requirement Engineering and D00RS

 

 

Yep, that version of the example script works.

Maybe I am stupid, but

>How should the parser know which variant the user wants?

Each AttrDef (ad variable) is of some specific AttrBaseType, so defval should return a value which belongs to some of the possible AttrBaseTypes. Do you mean that return value type is unspecified until string s = ad.defval forces it to be a string?

And how the heck not doing this type cast messes up DOORS?

But yes, I typically fetch such values into variables for further processing, as quite a while ago I myself noticed this kind of problem. Nice to know now that it has been designed into DOORS :-)

Re: Tool is working as designed?
Mathias Mamsch - Wed May 05 08:19:17 EDT 2010

SystemAdmin - Wed May 05 07:21:02 EDT 2010
Yep, that version of the example script works.

Maybe I am stupid, but

>How should the parser know which variant the user wants?

Each AttrDef (ad variable) is of some specific AttrBaseType, so defval should return a value which belongs to some of the possible AttrBaseTypes. Do you mean that return value type is unspecified until string s = ad.defval forces it to be a string?

And how the heck not doing this type cast messes up DOORS?

But yes, I typically fetch such values into variables for further processing, as quite a while ago I myself noticed this kind of problem. Nice to know now that it has been designed into DOORS :-)

This is built into the design of the DXL parser. The DXL parser uses the yacc library with a normal BNF grammar to parse the source. In this grammar you have rules like:

concatenation: expression IDENTIFIER
expression: binaryOperation
binaryOperation: IDENTIFIER DOT IDENTIFIER

When you have a statement like ad.defval "\n" the three rules are executed serially. First the binary operator (since it has a higher precedence than the concatenation): ad.defval At this point the parser needs to decide which version of ad.deval to call. It does not know at this point that there will be a concatenation next. So it will take the first or the last or a random one. Then it will recognize that a binaryOperation yields an expression, so it will take the result of the "." operator and store it as an expression. Then this expression is concatenated to the "\n" string. Again it has to decide which concatenation of an expression to use. And at this point the parser does not handle the type of the expression right and again decides randomly. I guess putting this type check into the parser is simply a huge load of work.

How does this mess stuff up? My guess is, that he will try to do a string concatenation, which will copy over data from a string until the end of the string ('\0') is reached. The problem is that he gets no string, but a Date with random data after it, so he goes on copying until he reaches a zero byte. While copying he will destroy data in the memory including viable data of the module that he stored there. The module is now unstable. If you do a "save module" in this situation you might destroy your module. So when you open the "edit attribute" dialog he tries to access some of the destroyed data yielding the exception.

Regards, Mathias

Mathias Mamsch, IT-QBase GmbH, Consultant for Requirement Engineering and D00RS

Re: Tool is working as designed?
SystemAdmin - Wed May 05 08:34:28 EDT 2010

Mathias Mamsch - Wed May 05 08:19:17 EDT 2010
This is built into the design of the DXL parser. The DXL parser uses the yacc library with a normal BNF grammar to parse the source. In this grammar you have rules like:

concatenation: expression IDENTIFIER
expression: binaryOperation
binaryOperation: IDENTIFIER DOT IDENTIFIER

When you have a statement like ad.defval "\n" the three rules are executed serially. First the binary operator (since it has a higher precedence than the concatenation): ad.defval At this point the parser needs to decide which version of ad.deval to call. It does not know at this point that there will be a concatenation next. So it will take the first or the last or a random one. Then it will recognize that a binaryOperation yields an expression, so it will take the result of the "." operator and store it as an expression. Then this expression is concatenated to the "\n" string. Again it has to decide which concatenation of an expression to use. And at this point the parser does not handle the type of the expression right and again decides randomly. I guess putting this type check into the parser is simply a huge load of work.

How does this mess stuff up? My guess is, that he will try to do a string concatenation, which will copy over data from a string until the end of the string ('\0') is reached. The problem is that he gets no string, but a Date with random data after it, so he goes on copying until he reaches a zero byte. While copying he will destroy data in the memory including viable data of the module that he stored there. The module is now unstable. If you do a "save module" in this situation you might destroy your module. So when you open the "edit attribute" dialog he tries to access some of the destroyed data yielding the exception.

Regards, Mathias

Mathias Mamsch, IT-QBase GmbH, Consultant for Requirement Engineering and D00RS

OK, that is then something we have to live with. Thanks Mathias for the information what's under the hood!

Re: Tool is working as designed?
llandale - Wed May 05 12:37:07 EDT 2010

Mathias has a good answer, especially his grasp of the internals.

I notice when I run your script, I get a bunch of Dec 31 1969 dates printed after all my attr names, then the crash. I notice if you make a single change to one line it works, calling out the string version of 'ad.defval'. 

print ad.name ": " (string ad.defval) "\n"


Don't know why it would trigger faults later; Mathias' guess seems reasonable.

I ran across this some years ago and wrote the following function which resolves the problem:

  

//********************
string    fDefVal(AttrDef ad)
{     // Retrieve the default value, if any, for the attribute.
        // Return null if it has no default value.
        // Returns 'True' or 'False' for Booleans that have a Default.
        // Programs can determine IF an attr has a default value like this:
        //      if (ad.defval){}
 
        if (null ad)    return("")    // Bad Input
        if (!ad.defval) return("")    // This attr has no default value
 
        string  Results = ad.defval
 
        return(Results)
        // Note: Consider a Boolean attribute that true has a default value of False.  
        // Since 'ad.defval' can mean either [1] does it have a default value,
        //      and [2] what that default value is, testing shows DXL should do this:
        //      [1] >> Use [print (bool ad.defval)]   or [bool IsDef = ad.defval] or [if (ad.defval)]
        //      [2] >> Use [print (string ad.defval)] or [string Value = ad.defval]
 
        // This also works, but not recommended:
        //      AttrDef ad = find(current Module, "aBool")
        //      string DefVal = ad.(ADADefault_ defval)
        //      bool   DefHas = ad.(ADABool_    defval)
        //      print DefVal "\t" DefHas "\n"
 
}     // end fDefVal()


Its verbose to house what I learned while trying to figure it out.

I find it VERY curious that the function doesn't have to cast 'ad.defval' when assigning to a string: Results = ad.defval; I presume that it accidentally finds the right "=(Attr__)" perm, probably by coincedence.

using "if (ad.defval)" works because by context it needs a 'boolean', and there is only one ad.defal which is boolean (whether the attr has a default value).

 

 

 

  • Louie


There was a similar issue when migrating to v8. A layout that did this: oOther = target(link), with AutoDeclare on, would cast oOther as an 'Object' in v7 but a 'ModName_' in v8, triggering DXL errors later when oOther was used.

 

 

Re: Tool is working as designed?
GerhardS - Tue Nov 16 09:30:11 EST 2010

Mathias Mamsch - Wed May 05 08:19:17 EDT 2010
This is built into the design of the DXL parser. The DXL parser uses the yacc library with a normal BNF grammar to parse the source. In this grammar you have rules like:

concatenation: expression IDENTIFIER
expression: binaryOperation
binaryOperation: IDENTIFIER DOT IDENTIFIER

When you have a statement like ad.defval "\n" the three rules are executed serially. First the binary operator (since it has a higher precedence than the concatenation): ad.defval At this point the parser needs to decide which version of ad.deval to call. It does not know at this point that there will be a concatenation next. So it will take the first or the last or a random one. Then it will recognize that a binaryOperation yields an expression, so it will take the result of the "." operator and store it as an expression. Then this expression is concatenated to the "\n" string. Again it has to decide which concatenation of an expression to use. And at this point the parser does not handle the type of the expression right and again decides randomly. I guess putting this type check into the parser is simply a huge load of work.

How does this mess stuff up? My guess is, that he will try to do a string concatenation, which will copy over data from a string until the end of the string ('\0') is reached. The problem is that he gets no string, but a Date with random data after it, so he goes on copying until he reaches a zero byte. While copying he will destroy data in the memory including viable data of the module that he stored there. The module is now unstable. If you do a "save module" in this situation you might destroy your module. So when you open the "edit attribute" dialog he tries to access some of the destroyed data yielding the exception.

Regards, Mathias

Mathias Mamsch, IT-QBase GmbH, Consultant for Requirement Engineering and D00RS

Matthias,

you mention a yacc library used for DXL. Is it publicly available (e.g., to be able to write code analysis tools)?

Regards,
Gerhard

Re: Tool is working as designed?
Mathias Mamsch - Tue Nov 16 10:01:17 EST 2010

GerhardS - Tue Nov 16 09:30:11 EST 2010
Matthias,

you mention a yacc library used for DXL. Is it publicly available (e.g., to be able to write code analysis tools)?

Regards,
Gerhard

Lex and Yacc are available under all modern programming environments (no DXL is no modern programming environment) like Java, Python, C/C++, etc. I personally like to use PLY (Python Lex Yacc) since I am a python fanatic and one can do complex parsing / code analysis tasks with a steep learning curve. However coming up with a grammar for a language like DXL is no thing that you do just like that :-)

However there are several other parser frameworks which are more modern and have better development environments. Take a look at ANTLRWORKS, which will enable you do create and debug grammars, without having to program.

Regards, Mathias
Mathias Mamsch, IT-QBase GmbH, Consultant for Requirement Engineering and D00RS

Re: Tool is working as designed?
llandale - Mon Mar 14 18:08:51 EDT 2011

Mathias Mamsch - Wed May 05 06:56:21 EDT 2010

Yeah one of the little DXL oddities.

You might not spot it right there, but the below script messes up DOORS, so the problem is not DOORS but the script below. There are a lot of ways to crash DOORS by a script, one needs to be careful when coding. In short: The problem is ambiguity!

Lets see why: The perms that are causing this mess are:
  

ADADefault_ defval ()
ADABool_ defval ()
 
real ::. (AttrDef, ADADefault_)
int ::. (AttrDef, ADADefault_)
string ::. (AttrDef, ADADefault_)
Date ::. (AttrDef, ADADefault_)
bool ::. (AttrDef, ADABool_)
 
string ::.. (real, string)
string ::.. (int, string)
string ::.. (string, string)
string ::.. (Date, string)
string ::.. (AttrBaseType, string)
string ::.. (bool, string)

 


The problematic thing about the script is the line:

 

  

print ad.name ": " ad.defval "\n"



This line can be read as various combinations of the above perms. defval can return either ADADefault or ADABool_. If defval returns ADADefault ad.deval can either be real, int, string or Date. If it returns ADABool_ it will return a bool (read that again). Ok this far?

So the string concatenation ad.deval "\n" will either be a real a bool, a date or a string concatenated to a string.

And now the question: How should the parser know which variant the user wants? It can't. At least he could choose a valid combination, that is if for ad.deval he chooses "Date" he should also call ..(Date, string) but that is where it goes wrong and messes up memory. It will let ad.defval return a Date or and then call the ::..(int, string) function to concatenate the Date to the string.

So after all, what is the solution? Code it like this and the crash goes away.



 

 

  

Module m = current
AttrDef ad = null
 
for ad in m do
   {
     if (ad.defval)
        {
        string s = ad.defval
            print ad.name ": " s "\n"
        }
   }


Hope this was understandable, the topic is a little awkward to explain. Regards, Mathias

 

 

 

 

Mathias Mamsch, IT-QBase GmbH, Consultant for Requirement Engineering and D00RS

 

 

... also, if the attribute is a boolean one then "ad.defval" will return either whether <1> it has a default value or <2> what that value is? However, string s = ad.defval will indeed what that default value is, either "True" if it has one and its true, or "False".

Anyway, had some code years ago messing up my DXL context such that no scripts would run, including those associated with native pull-down menus. Never figured it out and I'm wondering if this was the cause.

Hackles went up on my neck years ago when I saw two "defval" functions, both for use in "ad.defval". Said to myself "Disallowed Content Detected". That's like having two sons named "Daryl". Seems to me there's another one of them out there in DXL land...

  • Louie